Storage medium having input processing program stored thereon and input processing device

ABSTRACT

An input processing program for displaying a virtual 3-dimensional space and causing a computer to execute: a virtual plane setting step for setting a virtual plane in the virtual 3-dimensional space; a 2-dimensional coordinate detection step for detecting 2-dimensional coordinates inputted by a pointing device; an on-virtual plane moving step for moving a predetermined object on the virtual plane based on the 2-dimensional coordinates; an in-3-dimensional-space moving step for moving the object in the virtual 3-dimensional space out of the virtual plane, according to a predetermined input condition, and a display control step for displaying the object which is moved in the on-virtual plane moving step and the in-3-dimensional-space moving step and represented in the virtual 3-dimensional space.

CROSS RELATED APPLICATION

This application is a divisional application of U.S. patent applicationSer. No. 11/232,998 filed Sep. 23, 2005 and claims the benefit ofJapanese Patent Application No. 2004-304961 filed Oct. 19, 2004, theentirety of both applications are incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a storage medium having stored thereonan input processing program, and an input processing device. Moreparticularly, the present invention relates to a storage medium havingstored thereon an input processing program which is operated by using adevice for inputting 2-dimensional coordinates on a display screen to avirtual 3-dimensional space, and an input processing device.

2. Description of the Background Art

As conventional art, techniques operated by using a touch panel forinputting 2-dimensional coordinates on a display screen to a virtual3-dimensional space displayed on the display screen are disclosed in,for example, Japanese Laid-Open Patent Publication No. 11-7372 andJapanese Laid-Open Patent Publication No. 2004-70920. In any of thesetechniques, a virtual 3-dimensional space is displayed on a displayscreen and a touch panel or the like associated with the display screenis provided. And based on a position, on the touch panel, where a userpresses down, X and Y coordinates of the 3-dimensional space aredetermined, and based on the magnitude of a pressure at which the userpresses down on the touch panel, a Z coordinate of the 3-dimensionalspace (a depth direction) is determined.

In the conventional art described above, however, in order to detect themagnitude of a pressing force exerted on the touch panel or the like, itis necessary to additionally provide a function for detecting thepressing force, such as a pressure-sensitive element, which makes thedevice in itself complicated, resulting in cost increases. And when theuser enters a large input in the depth direction of the virtual3-dimensional space, the user is required to strongly press down on thetouch panel, leading to a heavy load exerted on the touch panel. Thiscauses the touch panel to easily break down or a shorter life thereof.

SUMMARY OF THE INVENTION

Therefore, in one embodiment, the present invention provides a storagemedium having stored thereon an input processing program in which basedon an input from a device for inputting 2-dimensional coordinates on adisplay screen, coordinates in a virtual 3-dimensional space areobtained, and an input processing device.

The reference numerals, step numbers and the like in the parenthesesindicate the correspondence with figures illustrated below in order toaid in understanding the present invention and are not to be construedas limiting, in any way, the scope of the present invention.

A first aspect of an embodiment of the present invention is directed toa storage medium having stored thereon a program executed by a computer(21) in an input processing device (1). The input processing devicecomprises a display screen (12) and a pointing device (13) for inputtingcorresponding 2-dimensional coordinates on the display screen. Theprogram causes the computer to execute a display control step (S57), a2-dimensional coordinate detection step (S54), a 2-dimensionalcoordinate shift amount calculation step (S72), and a 3-dimensionalcoordinate shift amount conversion step (S73). In the display controlstep, a virtual 3-dimensional space is displayed on the display screen(FIG. 3, FIG. 4A, and FIG. 4B). In the 2-dimensional coordinatedetection step, 2-dimensional coordinates inputted from the pointingdevice are detected. In the 2-dimensional coordinate shift amountcalculation step, shift amounts (vector v), per unit of time, of the2-dimensional coordinates detected in the 2-dimensional coordinatedetection step are calculated according to a predetermined calculationstart condition (Yes in S51, No in S52). In the 3-dimensional coordinateshift amount conversion step, the shift amounts calculated in the2-dimensional coordinate shift amount calculation step are converted to3-dimensional coordinate shift amounts (vector V) in the virtual3-dimensional space. The pointing device is an input device fordesignating 2-dimensional coordinates on the display screen, such as atouch panel, a mouse, a track pad, and a track ball. A coordinate systemused for each input device is a touch panel coordinate system or ascreen coordinate system.

In a second aspect based on the first aspect, the computer is furtheroperable to execute an input status determination step (S52). In theinput status determination step, a status inputted from the pointingdevice is determined. In the 2-dimensional coordinate shift amountcalculation step, based on the calculation start condition thatcalculation starts when a status where an input from the pointing deviceis being continuously conducted (Yes in S52) is changed to a statuswhere there is no input (No in S52) is determined in the input statusdetermination step, shift amounts, per unit of time, of the2-dimensional coordinates detected in the 2-dimensional coordinatedetection step immediately before the status of no input are calculated.

In a third aspect based on the second aspect, in the display controlstep, a predetermined virtual projection plane (S3 in FIG. 5) is set inthe virtual 3-dimensional space (S53) and when it is determined in theinput status determination step that the input from the pointing deviceis being continuously conducted, a predetermined object (I) is displayedat a position where the 2-dimensional coordinates detected in the2-dimensional coordinate detection step are projected on the virtualprojection plane (FIG. 3). In the display control step, when a3-dimensional coordinate shift amount conversion has been conducted inthe 3-dimensional coordinate shift amount conversion step, the objectis, based on the 3-dimensional coordinate shift amounts, lifted off thevirtual projection plane, moved in the virtual 3-dimensional space, anddisplayed therein (FIG. 4B).

In a fourth aspect based on the third aspect, in the display controlstep, when it is determined in the input status determination step thatan input from the pointing device is being. continuously conducted, adisplay angle (θ) of the object to be projected on the virtualprojection plane is controlled based on the 2-dimensional coordinatesdetected in the 2-dimensional coordinate detection step (S55, FIG. 8).

In a fifth aspect based on the third aspect, the computer is furtheroperable to execute a motion trajectory calculation step (S59). In themotion trajectory calculation step (S59), the 3-dimensional coordinateshift amounts converted in the 3-dimensional coordinate shift amountconversion step are set as an initial motion vector (V) of the object inthe virtual 3-dimensional space, and a motion trajectory, per unit oftime, in the virtual 3-dimensional space is calculated. In the displaycontrol step, based on the motion trajectory calculated in the motiontrajectory calculation step, the object is lifted off the virtualprojection plane, moved in the virtual 3-dimensional space, anddisplayed therein.

In a sixth aspect based on the fifth aspect, in the display controlstep, when it is determined in the input status determination step thatan input from the pointing device is being continuously conducted, adisplay angle of the object to be projected on the virtual projectionplane is controlled based on the 2-dimensional coordinates detected inthe 2-dimensional coordinate detection step. In the motion trajectorycalculation step, an initial normal vector (n) of the object is setaccording to the display angle, and a motion trajectory, per unit oftime, in the virtual 3-dimensional space is calculated based on themotion vector and the normal vector.

In a seventh aspect based on the first aspect, in the 3-dimensionalcoordinate shift amount conversion step, based on the shift amounts (vx,vy), of a first and a second axes, calculated in the 2-dimensionalcoordinate shift amount calculation step, a shift amount (Vz) of a thirdaxis perpendicular to the first and the second axes are calculated and a3-dimensional coordinate shift amount conversion is conducted.

In a eighth aspect based on the seventh aspect, in the 3-dimensionalcoordinate shift amount conversion step, when the shift amounts of thefirst and the second axes calculated in the 2-dimensional coordinateshift amount calculation step are vx and vy, respectively, andpredetermined constants are a, b, c, d, e, and f, a shift amount Vx ofthe first axis, a shift amount Vy of the second axis, and a shift amountVz of the third axis, which are represented as the 3-dimensionalcoordinate shift amounts, are calculated using:

$\begin{pmatrix}{Vx} \\{Vy} \\{Vz}\end{pmatrix} = {\begin{pmatrix}{ab} \\{cd} \\{ef}\end{pmatrix}\begin{matrix}{vx} \\{vy}\end{matrix}}$

In a ninth aspect based on the eighth aspect, constants a, b, c, d, e,and f respectively vary according to each kind of the objects (FIG. 7).

A tenth aspect is directed to a program which is executed by thecomputer in the input processing device. The input processing devicecomprises a display screen and a pointing device for inputtingcorresponding 2-dimensional coordinates on the display screen, and avirtual 3-dimensional space is displayed on the display screen. Theprogram causes the computer operable to execute a projection planesetting step (S53), a 2-dimensional coordinate detection step (S54), anon-projection-plane moving step (S54), an in-3-dimensional-space movingstep (S59), and a display control step (S57). In the projection planesetting step, the virtual projection plane is set in the virtual3-dimensional space. In the 2-dimensional coordinate detection step, the2-dimensional coordinates inputted from the pointing device are set. Inthe on-projection-plane moving step, by projecting on the virtualprojection plane the 2-dimensional coordinates detected in the2-dimensional coordinate detection step, a predetermined object is movedto a position on the virtual projection plane, corresponding to the2-dimensional coordinates. In the in-3-dimensional-space moving step,the object is moved in the virtual 3-dimensional space outside thevirtual projection plane, according to a predetermined input condition.In the display control step, the object which moves in theon-projection-plane moving step and the in-3-dimensional-space movingstep is represented in the virtual 3-dimensional space and displayed onthe display screen.

An eleventh aspect is directed to an input processing device comprisinga display screen, a pointing device, a display control means, a2-dimensional coordinate detection means, a 2-dimensional coordinateshift amount calculation means, and a 3-dimensional coordinate shiftamount conversion means. The pointing device inputs corresponding2-dimensional coordinates on the display screen. The display controlmeans displays the virtual 3-dimensional space on the display screen.The 2-dimensional coordinate detection means detects 2-dimensionalcoordinates inputted from the pointing device. The 2-dimensionalcoordinate shift amount calculation means, according to thepredetermined calculation start condition, calculates shift amounts, perunit of time, of the 2-dimensional coordinates detected by the2-dimensional coordinate detection means. The 3-dimensional coordinateshift amount conversion means converts the shift amounts calculated bythe 2-dimensional coordinate shift amount calculation means, to the3-dimensional coordinate shift amounts in the virtual 3-dimensionalspace.

In a twelfth aspect based on the eleventh aspect, the pointing device isa touch panel covering the display screen.

A thirteenth aspect is directed to an input processing device comprisinga display screen, a pointing device, a projection plane setting means, a2-dimensional coordinate detection means, an on-projection-plane movingmeans, an in-3-dimensional-space moving means, and a display controlmeans. The display screen displays a virtual 3-dimensional space. Thepointing device inputs corresponding 2-dimensional coordinates on thedisplay screen. The projection plane setting means sets a virtualprojection plane in the virtual 3-dimensional space. The 2-dimensionalcoordinate detection means detects the 2-dimensional coordinatesinputted from the pointing device. The on-projection-plane moving means,by projecting on the virtual projection plane the 2-dimensionalcoordinates detected by the 2-dimensional coordinate detection means,moves a predetermined object to a position on the virtual projectionplane, corresponding to the 2-dimensional coordinates. Thein-3-dimensional-space moving means, according to a predetermined inputcondition, moves the object in the virtual 3-dimensional space outsidethe virtual projection plane. The display control means represents inthe virtual 3-dimensional space the object which is moved by theon-projection-plane moving means and the in-3-dimensional-space movingmeans, and displays the object on the display screen.

In a fourteenth aspect based on the thirteenth aspect, the pointingdevice is a touch panel covering the display screen.

According to the first aspect, because the shift amounts of the2-dimensional coordinates are converted to the shift amounts of the3-dimensional coordinates according to the predetermined calculationstart condition, a simple configuration can achieve the conversion ofthe 2-dimensional coordinates to the 3-dimensional coordinates withoutproviding an extra input device of a pressing force detection functionor the like for obtaining 3-dimensional shift amounts. In addition,because of no detection of a pressing force exerted by a user, unlike inthe background art, a heavy burden on a pointing device such as a touchpanel is eliminated and a reduction in device reliability, which accruesfrom frequent breakdowns or a shorter life, can be avoided.

According to the second aspect, based on the condition that the statuswhere the input from the pointing device is being continuously conductedis changed to the status where there is no input, the 2-dimensionalcoordinates detected immediately before the status of no input areconverted to the 3-dimensional coordinates. Therefore, a simpleoperation allows the control by appropriately switching from the inputbased on the 2-dimensional coordinates to the input based on the3-dimensional coordinates in the virtual 3-dimensional space.

According to the third aspect, realized is an input processing whereaccording to 2-dimensional coordinates inputted from a pointing devicefor inputting 2-dimensional coordinates on a display screen, an objectmoves on a virtual projection plane and when the pointing device comesto input nothing, the object leaves the virtual projection plane andmoves in the virtual 3-dimensional space. For example, a game processingcan be realized where an item moves on a virtual projection plane set ina 3-dimensional game space while an input from a pointing device isbeing continuously conducted and the item is thrown from the virtualprojection plane to a game space when the pointing device comes to inputnothing.

According to the fourth aspect, a display angle of an object can becontrolled based on 2-dimensional coordinates inputted from a pointingdevice.

According to the fifth aspect, because a motion trajectory of an objectis shifted based on the converted 3-dimensional coordinate shiftamounts, a variety of motion trajectories can be displayed.

According to the sixth aspect, because a motion trajectory of an objectis shifted further based on a normal vector obtained from a displayangle, of the object, which varies according to 2-dimensionalcoordinates, a variety of motion trajectories according to positionsdesignated by a pointing device can be displayed.

According to the seventh aspect, because a third axis componentperpendicular to 2 axes composing a 2-dimensional coordinate system iscalculated based on 2-dimensional coordinate shift amounts,3-dimensional coordinate shift amounts can be easily obtained from the2-dimensional shift amounts.

According to the eighth aspect, when 3-dimensional coordinate shiftamounts are calculated from 2-dimensional shift amounts, shift amountsof respective axes can be easily obtained using determinants.

According to the ninth aspect, because 3-dimensional shift amountsaccording to a kind of objects can be obtained, wide variations inmotion control of the object in a virtual 3-dimensional space can beattained.

According to the tenth aspect, an input control can be realized underwhich an object moves on a virtual projection plane according tocoordinates inputted from a pointing device for inputting 2-dimensionalcoordinates on a display screen and the object moves from a virtualprojection plane to a virtual 3-dimensional space according to thepredetermined input condition.

In addition, the input control device enables the same effect as that ofthe aforementioned storage medium having stored thereon the inputcontrol program.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an outline view illustrating a game apparatus 1 executing agame program according to one embodiment of the present invention;

FIG. 2 is a block diagram illustrating the game apparatus 1 shown inFIG. 1;

FIG. 3 shows an example of a display screen image on the second LCD 12,illustrating a view of determining an initial position of an item I tobe thrown in a game space;

FIG. 4A and FIG. 4B show examples of display screen images on the secondLCD 12, illustrating views of operations of throwing the item I in thegame space and of moving the thrown item I in the game space;

FIG. 5 is a conceptual diagram illustrating a virtual 3-dimensional gamespace and a virtual projection plane;

FIG. 6A and FIG. 6B are conceptual diagrams illustrating a vector v (vx,vy) and a vector V (Vx, Vy, Vz);

FIG. 7 shows an example of setting of constants a to f used for acoordinate conversion;

FIG. 8 shows an example of a screen display image of the item Iaccording to a tilt angle which is initially set on the item I;

FIG. 9 is a conceptual diagram of a motion vector and a normal vectorwhich are set when the item I leaves the virtual projection plane andmoves in the virtual game space;

FIG. 10 is a flow chart illustrating an operation conducted by the gameapparatus 1 by executing the game program according to the presentinvention; and

FIG. 11 is a flow chart illustrating an operation conducted by the gameapparatus 1 by executing the game program according to the presentinvention.

DESCRIPTION OF THE PREFERRED

A game apparatus which executes a game program will be described withreference to the figures. FIG. 1 is an outline view showing an outerappearance of the game apparatus 1 which executes a game program. As anexample of the game apparatus 1, a hand-held type game apparatus isillustrated herein. And a game program used in the following explanationis an example of an input processing program of the present inventionand a game apparatus 1 used in the following explanation is an exampleof an input processing apparatus of the present invention.

In FIG. 1, the game apparatus 1 of the present embodiment isaccommodated in a housing 18 so that two liquid crystal display devices(hereinafter referred to as “LCDs”) 11 and 12 are placed inpredetermined positions. Specifically, in the case where the first LCD11 and the second LCD 12 are to be disposed one on top of the other, thehousing 18 is composed of a lower housing 18 a and an upper housing 18b, the upper housing 18 b being supported by a portion of the upper sideof the lower housing 18 a so as to be pivotable. The upper housing 18 bhas a planar contour which is slightly larger than that of the first LCD11. The upper housing 18 b has an opening in one principal face thereof,through which a display screen of the first LCD 11 is exposed. The lowerhousing 18 a has a more elongated planar contour than that of the upperhousing 18 b (i.e., so as to have a longer lateral dimension). Anopening for exposing the display screen of the second LCD 12 is formedin a portion of the lower housing 18 a which lies substantially in thecenter of the lower housing 18 a along the lateral direction. A soundhole for the loudspeaker 15 is formed in either (right or left) wing ofthe lower housing 18 a between which the second LCD 12 is interposed. Anoperation switch section 14 is provided on the right and left wings ofthe lower housing 18 a between which the second LCD 12 is interposed.

The operation switch section 14 includes: an operation switch (“A”button) 14 a and an operation switch (“B” button) 14 b, which areprovided on a principal face of the right wing of the lower housing 18 a(lying to the right of the second LCD 12); and a direction switch (crosskey) 14 c, a start switch 14 d, a select switch 14 e, and side switches14 f and 14 g, which are provided on a principal face of the left wingof the lower housing 18 a (lying to the left of the second LCD 12). Theoperation switches 14 a and 14 b are used for giving instructions suchas: “pass”, “shoot”, etc., in the case of a sports game such as a soccergame; “jump”, “punch”, “use a weapon”, etc., in the case of an actiongame; or “get an item”, “select a weapon”, “select a command”, etc., inthe case of a role playing game (RPG) or a simulation RPG. The directionswitch 14 c is used by a player for providing instructions concerningdirections on the game screen, e.g., instructions of a moving directionfor (i.e., a direction in which to move) a player object (or a playercharacter) that can be controlled by using the operation switch section14, or instructions of a moving direction for a cursor, for example. Theside switches (“L” button) 14 f and (“R” button) 14 g are provided atthe left and right ends of an upper face (upper side face) of the lowerhousing 18 a. As necessary, more operation switches may be added.

A touch panel 13 (an area marked by dotted lines in FIG. 1) is mountedon the upper principal face of the second LCD 12. The touch panel 13 maybe of any one of a resistive film type, an optical type (infrared type),or a capacitive coupling type. When a stylus 16 (or a finger) is pressedagainst or moved or dragged on the upper principal face of the touchpanel 13, the touch panel 13 detects the coordinate position of thestylus 16 and outputs coordinate data.

As necessary, a hole (an area marked by double-dot lines in FIG. 1) foraccommodating the stylus 16 with which to manipulate the touch panel 13is provided near a side face of the upper housing 18 b. The hole canhold the stylus 16. In a portion of a side face of the lower housing 18a is provided a cartridge receptacle (an area marked by dash-dot linesin FIG. 1), in which a game cartridge 17 (hereinafter simply referred toas “the cartridge 17”) internalizing a memory having a game programstored therein (e.g., a ROM) is detachably inserted. The cartridge 17 isan information storage medium for storing a game program, e.g., anon-volatile semiconductor memory such as a ROM or a flash memory. Aconnector (see FIG. 2) lies inside the cartridge receptacle forproviding electrical connection with the cartridge 17. Furthermore, thelower housing 18 a (or alternatively the upper housing 18 b)accommodates an electronic circuit board on which various electroniccomponents such as a CPU are mounted. Examples of the informationstorage medium for storing a game program are not limited to theaforementioned non-volatile semiconductor memory, but may also be aCD-ROM, a DVD, or any other optical disk type storage medium.

FIG. 2 is a block diagram illustrating an internal structure of the gameapparatus 1 of FIG. 1. In FIG. 2, a CPU core 21 is mounted on theelectronic circuit board 20 accommodated in the housing 18. Via a givenbus, the CPU core 21 is connected to a connector 28, an input/outputinterface (I/F) circuit 27, a first graphics processing unit (first GPU)24, a second graphics processing unit (second GPU) 26, a WRAM 22, and anLCD controller 29. The cartridge 17 is detachably connected to theconnector 28. The cartridge 17 is a storage medium for storing a gameprogram, and specifically, the cartridge 17 includes a ROM 171 forstoring a game program and a RAM 172 for storing backup data in arewritable manner. A game program which is stored in the ROM 171 of thecartridge 17 is loaded to a WRAM 22, and the game program having beenloaded to the WRAM 22 is executed by the CPU core 21. Temporary datawhich is obtained by the CPU core 21 executing the game program and datafrom which to generate images are stored in the WRAM 22. The I/F circuit27 is connected to the operation switch section 14, the touch panel 13,and the loudspeaker 15.

The first GPU 24 is connected to a first video-RAM (a first VRAM) 23.The second GPU 26 is connected to a second video-RAM (a second VRAM) 25.In accordance with an instruction from the CPU core 21, the first GPU 24generates a first game image on the basis of the data used for imagegeneration which is stored in the WRAM 22, and writes (stores) images inthe first VRAM 23. In accordance with an instruction from the CPU core21, the second GPU 26 generates a second game image on the basis of thedata used for image generation which is stored in the WRAM 22, andwrites (stores) images in the second VRAM 25. The first VRAM 23 and thesecond VRAM 25 are connected to an LCD controller 29.

The LCD controller 29 includes a register 291. The register 291 stores avalue of 0 or 1 in accordance with an instruction from the CPU core 21.If a value of the register 291 is 0, the LCD controller 29 outputs agame image written in the first VRAM 23 to the first LCD 11 and a gameimage written in the second VRAM 25 to the second LCD 12. And if a valueof the register 291 is 1, the LCD controller 29 outputs a game imagewritten in the first VRAM 23 to the second LCD 12 and a game imagewritten in the second VRAM 25 to the first LCD 11.

The I/F circuit 27 is a circuit which controls exchanges of data betweenthe CPU core 21 and the external input/output devices such as, theoperation switch section 14, the touch panel 13, and the loudspeaker 15.The touch panel 13 (including a device driver for the touch panel) has acoordinate system corresponding to the coordinate system of the secondVRAM 25, and outputs data of position coordinates corresponding to aposition which is input (designated) by means of the stylus 16. Thedisplay screen of the second LCD 12 has a resolution of 256 dots ×192dots, and the touch panel 13 also has a detection accuracy of 256 dots×192 dots so as to correspond to the display screen. The detectionaccuracy of the touch panel 13 may be lower or higher than theresolution of the display screen of the second LCD 12.

Hereinafter, referring to FIG. 3, FIG. 4A, and FIG. 4B, a flow of gameprocessing of the game program executed by the game apparatus 1 will bedescribed with reference to examples of specific display screen images.While in an embodiment of the present invention, a game, in which itemsare thrown in a game space, executed by the game apparatus 1, will bedescribed, the description for this kind of the game is not to beconstrued as limiting the present invention. FIG. 3 shows an example ofa display screen image of the second LCD 12 illustrating how an initialposition of an item I to be thrown in a game space is determined. FIG.4A shows an example of a display screen image on the second LCD 12,illustrating an operation of throwing the item I in the game space. FIG.4B shows an example of a display screen image on the second LCD 12,illustrating an operation of moving the thrown item I in the game space.

In FIG. 3, a view of a virtual 3-dimensional space is displayed on thesecond LCD 12, and the item I (a flying disc is shown in FIG. 3) whichis thrown in the game space is displayed. As is made clear by the belowdescription, the game space corresponding to a silhouette volume basedon a given camera view point is displayed on the second LCD 12 and theitem I is projected on a virtual projection plane which is set withinthe silhouette volume. A player can move the item I in the game space bytouch-operating a position of the item I displayed on the second LCD 12by means of the touch panel 13. Specifically, when the player conducts atouch-operation dragging the item I displayed on the second LCD 12 bymeans of the touch panel 13 (touching a position of the touch panel 13superimposed on the item I on the second LCD 12, keeping as it is, andthen moving the touch-operating position), the item I moves to aposition, of the virtual projection plane, which corresponds tocoordinates inputted from the touch panel 13. For example, FIG. 3 showsan example in which the player moves the item I in A direction asillustrated. In other words, the player can move the item I on thevirtual projection plane through the touch-operation of dragging theitem.

When the player finishes the touch-operation on the touch panel 13 afterthe touch-operation dragging the item I (that is, when the player liftsoff the touch panel the stylus 16 or the like being used for thetouch-operation), the item I is thrown in the game space from thevirtual projection plane. Suppose that as shown in FIG. 4A, the playerconducts a touch-operation dragging the item I in direction B andfinishes the touch-operation at a point C by lifting the stylus 16 orthe like off the touch-panel 13. In this case, based on 2-dimensionalcoordinate information inputted from the touch panel 13 immediatelybefore finishing the touch-operation, the item I is thrown in the gamespace from the virtual projection plane. As shown in FIG. 4B, based on2-dimensional coordinate information (vector B) inputted from the touchpanel 13 immediately before finishing the touch-operation at the pointC, 3-dimensional coordinate information (motion vector D) which is setin a virtual 3-dimensional game space is calculated, and based on themotion vector D, the item I leaves the virtual projection plane andmoves in the game space.

Next, referring to FIG. 5, a virtual 3-dimensional game space and avirtual projection plane displayed on the second LCD 12 will bedescribed. FIG. 5 is a conceptual diagram illustrating the virtual3-dimensional game space and the virtual projection plane.

In FIG. 5, a front clip plane S1 and a rear clip plane S2 with referenceto a camera view point P are set. A space set in a silhouette volumesandwiched between the front clip plane S1 and the rear clip plane S2 isdisplayed on the second LCD 12. A virtual projection plane S3 is setwithin the silhouette volume and placed, for example, in parallel withthe front clip plane S1. A coordinate system of the touch panel is seton the front clip plane S1 and coordinates inputted from the touch panel13 are projected on the virtual projection plane S3. Although in orderto facilitate the understanding of the present invention, the front clipplane S1 on which the touch panel coordinate system is set and thevirtual projection plane S3 are arranged in parallel with each other,needless to say, projection of the input coordinates can be conducted inthe same manner even if the front clip plane S1 and the virtualprojection plane S3 are not in parallel with each other. Also needlessto say, the projection of the input coordinates can be conducted in thesame manner even if the virtual projection plane S3 is of a sphere orthe like, not a plane.

Next, referring to FIG. 6A, FIG. 6B, and FIG. 7, a coordinate conversionfrom the touch panel coordinate system displayed in 2-dimensionalcoordinates to a game space coordinate system displayed in a3-dimensional coordinate system will be described. FIG. 6A is aconceptual diagram illustrating a vector v (vx, vy) to be set in thetouch panel coordinate system which is set on the touch panel 13. FIG.6B is a conceptual diagram illustrating a vector V (Vx, Vy, Vz) to beset in 3-dimensional coordinates which is set in the virtual3-dimensional game space. FIG. 7 shows an example of setting ofconstants a to f used for the coordinate conversion.

In FIG. 6A, if the touch panel 13 is touch-operated from a point q1 (x1,y1) to a point q2 (x2, y2) in the touch panel coordinate system, avector v (vx, vy) spanning from the point q1 to the point q2 is obtainedas follows.

vx=x2−x1

vy=y2−y1

In the present embodiment, when the player conducts an operation on thetouch panel 13, corresponding to a predetermined condition, theaforementioned 2-dimensional vector v (vx, vy) set immediately beforethe operation is conducted is coordinate-converted, and a 3-dimensionalvector v (Vx, Vy, Vz) as shown in FIG. 6B is calculated. Here, thecoordinate conversion from the 2-dimensional vector v (vx, vy) to the3-dimensional vector V (Vx, Vy, Vz) is conducted as follows.

$\begin{pmatrix}{Vx} \\{Vy} \\{Vz}\end{pmatrix} = {\begin{pmatrix}{ab} \\{cd} \\{ef}\end{pmatrix}\begin{matrix}{vx} \\{vy}\end{matrix}}$

Constants a to f used in the above formula are set for each item thrownin the game space as shown in FIG. 7. For example, if an item I is aflying disc, constants are set as follows: a=1.0, b=0, c=0, d=0.5, e=0,and f=2.0, and a vector V (Vx, Vy, Vz) is calculated by using Vx=vx,Vy=0.5vy, Vz=2vy. As described above, in the coordinate conversion ofthe present embodiment, values of respective 3 axes of motion amounts(vector V) which are set in a 3-dimensional coordinate system arecalculated by using the values of the respective 2 axes, represented asmotion amounts (vector v) between two points, which are set in a2-dimensional coordinate system. And because the constants used for thecoordinate conversion vary depending on each item thrown in the gamespace, characteristics of each item can be represented through thecoordinate conversion.

Next, referring to FIG. 8 and FIG. 9, calculation of a motion trajectorywhen an item I leaves a virtual projection plane and moves in a virtualgame space will be described. FIG. 8 shows an example of a screendisplay image of the item I according to a tilt angle which is initiallyset on the item I. FIG. 9 is a conceptual diagram illustrating a motionvector and a normal vector which are set when the item I leaves thevirtual projection plane and moves in the virtual game space.

In FIG. 8, when a user conducts a touch-operation dragging the item Idisplayed on the second LCD 12 by using the touch panel 13, as describedabove, the item I moves to a position on the virtual projection planeaccording to coordinates inputted from the touch panel 13. And a tiltangle θ which is initially set on the item I is set based on a positionin an xm direction on the touch panel (lateral direction in FIG. 8), asshown in the figure. Specifically, let a center of the touch panel, aright direction and a left direction be xm=0, +xm, and −xm,respectively. And the tilt angle θ is obtained as follows.

θ=jxm+k (j and k are constants)

A tilt angle in an upward direction in FIG. 8, which is a verticaldirection on the touch panel 13, is θ=0°. And on the second LCD 12, theitem I is displayed by adjusting the tilt angle θ of the normal thereof.In other words, the item I is displayed by tilting the item I based on atilt angle corresponding to a position in the horizontal direction onthe touch panel. And based on the initially set tilt angle θ of the itemI, a normal vector n=(sin θ, cos θ, 0) is initially set.

As described above, after an operation where the item I leaves thevirtual projection plane and is thrown in the 3-dimensional game space,the item I moves in the game space based on the motion vector and thenormal vector. In FIG. 9, a motion vector and a normal vector which areset on each item I are calculated for each frame in which gameprocessing is conducted. Specifically, a normal vector n (i+1) in a newframe (i+1) is calculated using a normal vector n (i) set in a frame (i)immediately preceding, a motion vector V (i), and a constant a asfollows,

${\overset{\_}{n}\left( {i + 1} \right)} = \frac{{\overset{\_}{n}(i)} + {\alpha \; {V_{xz}(i)}}}{{{\overset{\_}{n}(i)} + {\alpha \; {\overset{\_}{V_{xz}}(i)}}}}$

where V_(xz) is a motion vector on an XZ plane when Y is 0. And a motionvector V (i+1) in a new frame (i+1) is calculated using a normal vectorn (i) set in a frame (i) immediately preceding, a motion vector V (i), agravity vector g, and a constant β as follows,

V (i+1)= V (i)+β| V (i)| n _(xz) (i)+g

where n_(xz) is a normal vector on an XZ plane when Y is 0.

Next, referring to FIG. 10 and FIG. 11, processes, based on informationinputted from the touch panel 13, which are executed by the gameapparatus 1 according to a game program will be described. FIG. 10 andFIG. 11 are flow charts illustrating operations which are conducted bythe game apparatus 1 by executing the game program. The programs forexecuting these processes are contained in the game program which isstored in the ROM 171 and are loaded from the ROM 171 to the WRAM 22when power of the game apparatus 1 is turned on, so as to be executed bythe CPU core 21.

When the power source (not shown) of the game apparatus 1 is turned on,the CPU core 21 executes a boot program (not shown), and thereby thegame program stored in the cartridge 17 is loaded to the WRAM 22. Thegame program having been loaded is executed by the CPU core 21, therebyto execute steps (abbreviated as “S” in FIGS. 10 and 11) shown in FIG.10 and FIG. 11. The game program is executed, and thereby game imagesand the like in accordance with the game program are written on thefirst LCD 11 and the second LCD 12. The detailed description of thecontents of the game is not given. Here, the processes in which the itemmoves according to the information inputted from the touch panel 13 willbe described in detail.

In FIG. 10, the CPU core 21 starts the game processing, and after eachkind of initialization, starts the game. The CPU core 21 determineswhether or not an item designation flag is on (step 51). The CPU core 21proceeds to the next step 52 when the item designation flag is on, andproceeds to the next step when the item designation flag is off. Here,the item designation flag is a flag to determine whether or not a playeris touching an item I (see FIG. 3) by means of the touch panel 13, andis set so as to be turned on when the player is touching an item I.

In step 52, the CPU core 21 determines whether or not there is an inputfrom the touch panel 13. And the CPU core 21 proceeds to the next step53 when there is an input from the touch panel 13, and proceeds to thenext step 71 when there is no input from the touch panel 13.

In step 53, the CPU core 21 sets a virtual projection plane S3 (see FIG.9) in a virtual 3-dimensional game space, and proceeds to the next step.Since the virtual projection plane S3 is as described above, thedetailed description is not given here.

Next, the CPU core 21 detects coordinates inputted from the touch panel13 and adjusts a display position of the item I to a 2-dimensionalcoordinate position on the virtual projection plane S3 corresponding tothe detected coordinates (step 54; FIG. 3). The CPU core 21 calculates atilt angle θ of the item I, based on an x coordinate value (a lateralcoordinate on the touch panel 13; an xm direction shown in FIG. 8) ofthe inputted coordinates (step 55; FIG. 8). The CPU core 21 calculates anormal vector n in initial setting of the item I, based on the tiltangle θ calculated in the above step 55 (step 56). Here, the CPU core 21calculates the normal vector n of the item I using n=(sin θ, cos θ, 0).The CPU core 21 tilts the item I according to the tilt angle θcalculated in step 55, conducts processes of item display control (step57; FIG. 8.) for the second LCD 12, and when the game is continued (Noin step 63), returns to the above step 51 to repeat the processes. TheCPU core 21 repeats these steps 51 to 57, and thereby the item I moveson the virtual projection plane S3 according to the touch-operation onthe touch panel 13 conducted by the player.

On the other hand, referring to FIG. 11, processes in which the itemdesignation flag is on (Yes in step 51) and there is no input from thetouch panel 13 (No. in step 52) will be described. When it is determinedin step 52 that there is no input from the touch panel 13, the CPU core21 determines whether or not there is an input from the touch panel 13in 2 frames immediately preceding (step 71). When there is an input fromthe touch panel 13 in the 2 frames immediately preceding, the CPU core21 proceeds to the next step 72. When there is no input from the touchpanel 13 in either one of the 2 frames immediately preceding, the CPUcore 21 proceeds to the next step 63.

In step 72, the CPU core 21 calculates coordinate shift amounts betweenthe 2 frames immediately preceding, using respective coordinatesinputted from the touch panel 13. Specifically, when the inputcoordinates of the 2 frames immediately preceding are a point q1 (x1,y1) and a point q2 (x2, y2), a vector v spanning from the point q1 tothe point q2 (vx, vy) is calculated as the coordinate shift amounts asfollows.

vx=x2−x1

vy=y2−y1

And the CPU core 21 proceeds to the next step.

Next, the CPU core 21 calculates a motion velocity (motion vector V) ofthe item I in the virtual 3-dimensional game space (step 73) based onthe coordinate shift amounts (vector v) obtained in the above step 72,and proceeds to the next step. In step 73, conducted is a coordinateconversion where values of respective 3 axes of the motion amounts(vector V) which are set in the 3-dimensional coordinate system arecalculated by using values of respective 2 axes represented as themotion amounts (vector v) between 2 points which are set in theaforementioned 2-dimensional coordinate system. Here, constants a to fare set according to a kind of the items I as described above.

Next, the CPU core 21 turns off the item designation flag (step 74) andproceeds to step 57. The CPU core 21 executes these steps 71 to 74 andthereby the motion amounts between the 2 points, which are set in the2-dimensional coordinate system, are coordinate-converted to the motionamounts (vector V) which are set in the 3-dimensional coordinate system.

Referring back to FIG. 10, processes in which the item designation flagis off (No in step 51) will be described. When it is determined in step51 that the item designation flag is off, the CPU core 21 determineswhether or not the item I has left the virtual projection plane and ismoving in the 3-dimensional game space (step 58). Here, the CPU core 21determines whether the item I is moving in the 3-dimensional game space,for example when a motion vector V is set on the item I. And when theitem I is moving in the game space, the CPU core 21 proceeds to the nextstep 59. When the item I is not moving in the game space (for example,when the player first touches the touch panel 13 or when the player isnot touching the touch panel 13 at all), the CPU core 21 proceeds to thenext step 60.

In step 59, the CPU core 21 calculates a motion trajectory of the item Iin the 3-dimensional game space. For the calculation of the motiontrajectory of the item I, the CPU core 21 calculates a normal vector nand a motion vector V of a new frame, using the normal vector n and themotion vector V calculated in the frames immediately preceding, asdescribed above (see FIG. 9). And the CPU core 21 moves the item Iaccording to the normal vector n and the motion vector V calculated instep 59, conducts processes of item display control on the second LCD(step 57; FIG. 4B), and when the game is continued (No in step 63),returns to the above step 51 and repeats the processes. The CPU core 21repeats these steps 51, 58, 59 and 57, and thereby the item I whichleaves the virtual projection plane and moves in the virtual game spaceis represented.

In step 60, the CPU core 21 determines whether or not there is an inputfrom the touch panel 13. The CPU core 12 proceeds to the next step 61when there is an input from the touch panel 13, and proceeds to the nextstep 63 when there is no input from the touch panel 13.

In step 61, the CPU core 21 determines whether or not the player istouch-operating a portion of the touch panel 13 where the item I issuperimposed on the second LCD 12. When the player is touch-operatingthe item I, the CPU core 21 turns on the item designation flag (step 62)and proceeds to step 63. When the player is not touch-operating the itemI, the CPU core 21 proceeds directly to step 63.

In step 63, the CPU core determines whether or not the game iscontinued. The CPU core returns to the above step 51 and repeats theprocesses when the game is continued, and ends the processing of thissubroutine when the game is finished. The processing of the above steps51 to 63 is repeated per unit of time (for example, one frame) for gameprocessing.

Although in the above description, the motion vector V for the virtual3-dimensional game space is calculated based on the condition that theplayer lifts the stylus 16 or the like off the touch panel 13 (No instep 52), the calculation may be conducted based on other conditions.For example, the calculation of the motion vector V may be conductedbased on a condition that a player presses down the operation switchsection (for example, the operation switch (A button) 14 a

Although in the above description, the virtual projection plane S3 isdescribed as a plane which is placed in parallel with the front clipplane S1, the virtual projection plane S3 and the front clip plane S1may be in non-parallel with each other. Even if the virtual projectionplane S3 is inclined toward the front clip plane S1, 2-dimensionalcoordinates (X axis, Y axis) on the virtual projection S3 can be set bysimilarly conducting the projection of input coordinates. In this case,with a direction perpendicular to the virtual projection plane S3 as thethird axis (Z axis), the motion vector V in the virtual 3-dimensionalgame space can similarly be calculated using the aforementionedcoordinate conversion.

Thus, according to the game apparatus disclosed herein, realized is agame in which an item moves according to coordinates inputted from atouch panel for inputting 2-dimensional coordinates on a display screen,and the item is thrown in a virtual 3-dimensional game space from avirtual projection plane based on a predetermined condition (anoperation of lifting off the touch panel). In addition, becausecomponents perpendicular to the virtual projection plane are calculatedbased on shift amounts (vector v) of the 2-dimensional coordinates whichare set on the virtual projection plane, shift amounts (vector V) of the3-dimensional coordinates can easily be obtained from the shift amountsof the 2-dimensional coordinates. Therefore, a simple configuration canachieve a conversion from 2-dimensional coordinates to 3-dimensionalcoordinates without providing a pressing force detection function,unlike in the conventional art. And because of no pressing forcedetection, a heavy burden on an input means such as a touch panel or thelike is eliminated and a reduction in device reliability, which accruesfrom frequent breakdowns or a shorter life, can be avoided.

Although in the above embodiment, the touch panel is used as an inputdevice for inputting 2-dimensional coordinates on a display screen,other pointing devices may be used. Here, a pointing device is an inputdevice for designating input positions and coordinates on a displayscreen, and when a mouse, a track pad, or a track ball, for example, isused as an input device and information of a screen coordinate systemcalculated from values outputted from the input device is used, thepresent invention can similarly be realized. In the case where apointing device such as a mouse or the like is used, processing forcalculating coordinates from values outputted from a mouse or the likemay be conducted on a game apparatus or the like, with a touch statusand a non-touch status corresponding to on and off of a click button.

Needless to say, although in the present embodiment, the touch panel 13is mounted on the game apparatus 1 in an integrated manner, even aconfiguration where a game apparatus and a touch panel are placed in aseparated manner can realize the present invention. And although in theabove embodiment, two display devices are provided, one display devicemay be applicable. In other words, in the above embodiment, only thesecond LCD 12 may be mounted without providing the first LCD 11. And inthe above embodiment, the touch panel 13 may be attached on the uppersurface of the first LCD 11 without providing the second LCD 12.

In addition, although in the above embodiment, the touch panel 13 ismounted on the game apparatus 1 in an integrated manner, an informationprocessing device such as a general personal computer or the like wherea touch panel is used as an input device may also be applicable. In thiscase, a program which the computer of the information processing deviceexecutes is not limited to a game program typically used for a game, andis a general-purpose input processing program where 2-dimensionalcoordinate values obtained by the aforementioned method are used inoperation processing for the above information processing device.

A storage medium having an input processing program stored thereon andan input processing device enable, with a simple configuration, aconversion from 2-dimensional coordinates to 3-dimensional coordinates,and are applicable to games and input processing or the like where apointing device for inputting 2-dimensional coordinates on a displayscreen is used.

While the invention has been described in detail, the foregoingdescription is in all aspects illustrative and not restrictive. It isunderstood that numerous other modifications and variations can bedevised without departing from the scope of the invention.

1. A non-transitory storage medium having stored thereon an inputprocessing program executed by a computer in an input processing devicefor displaying a virtual 3-dimensional space, including a display screenand a pointing device for inputting corresponding 2-dimensionalcoordinates on the display screen, the input processing program causingthe computer to execute: a virtual plane setting step for setting avirtual plane in the virtual 3-dimensional space; a 2-dimensionalcoordinate detection step for detecting the 2-dimensional coordinatesinputted from the pointing device; an on-virtual plane moving step formoving a predetermined object on the virtual plane based on the2-dimensional coordinates detected in the 2-dimensional coordinatedetection step; an in-3-dimensional-space moving step for moving theobject in the virtual 3-dimensional space out of the virtual plane,according to a predetermined input condition, and a display control stepfor displaying, on the display screen, the object which is moved in theon-virtual plane moving step and the in-3-dimensional-space moving stepand represented in the virtual 3-dimensional space.
 2. Thenon-transitory storage medium having stored thereon the input processingprogram according to claim 1 wherein the display screen is a touchsensitive display screen.
 3. The non-transitory storage medium havingstored thereon the input processing program according to claim 1 whereinthe 2-dimensional coordinates inputted from the pointing device areinputted as a series of coordinates inputted as the pointing device isdragged across the display screen.
 4. The non-transitory storage mediumhaving stored thereon the input processing program according to claim 1wherein the virtual plane is a two-dimensional plane
 5. Thenon-transitory storage medium having stored thereon the input processingprogram according to claim 1 wherein the virtual plane is on a virtualcurved surface.
 6. The non-transitory storage medium having storedthereon the input processing program according to claim 1 wherein thepredetermined input condition is lifting the pointing device from thedisplay screen.
 7. The non-transitory storage medium having storedthereon the input processing program according to claim 1 wherein thein-3-dimensional-space moving step occurs after the on-virtual planemoving step.
 8. The non-transitory storage medium having stored thereonthe input processing program according to claim 1 wherein the on-virtualplane moving step includes moving the predetermined object to a positionon the virtual plane corresponding to the 2-dimensional coordinatesdetected in the 2-dimensional coordinate detection step.
 9. Thenon-transitory storage medium having stored thereon the input processingprogram according to claim 1 wherein the on-virtual plane moving stepincludes moving the predetermined object to positions on the virtualplane corresponding to a continuous series of the 2-dimensionalcoordinates detected in the 2-dimensional coordinate detection step, andthe in-3-dimensional-space moving step begins when the 2-dimensionalcoordinate detection step detects the pointing device ceasing the inputof the 2-dimensional coordinates.
 10. The non-transitory storage mediumhaving stored thereon the input processing program according to claim 9wherein in the display control step includes changing a display angle ofthe predetermined object projected on the virtual plane when an inputfrom the pointing device based on 2-dimensional coordinates continuouslyinputted by the pointing device.
 11. The non-transitory storage mediumhaving stored thereon the input processing program according to claim 9wherein when an input from the pointing device is determined as beingcontinuously inputted, a display angle of the predetermined objectprojected on the virtual plane is controlled based on the 2-dimensionalcoordinates detected in the 2-dimensional coordinate detection step, andin the in-3-dimensional-space moving step, an initial normal vector ofthe predetermined object is set according to the display angle, and amotion trajectory in the virtual 3-dimensional space is calculated basedon the initial normal vector, wherein the movement of the object in thevirtual 3-dimensional space is based on the motion trajectory.
 12. Aninput processing device, comprising: a display screen adapted to displaya projection of a virtual 3-dimensional space; a pointing device adaptedto input corresponding 2-dimensional coordinates on the display screen;a virtual plane setting means for setting a virtual plane in the virtual3-dimensional space; a 2-dimensional coordinate detection means fordetecting the 2-dimensional coordinates inputted from the pointingdevice; an on-virtual plane moving means for moving a predeterminedvirtual object in the virtual plane based on the 2-dimensionalcoordinates detected by the 2-dimensional coordinate detection means; anin-3-dimensional-space moving means for moving the predetermined virtualobject in the virtual 3-dimensional space out of the virtual planeaccording to a predetermined input condition, and a display controlmeans for displaying on the display screen the movements of thepredetermined virtual object by the on-virtual plane moving means andthe in-3-dimensional-space moving means.
 13. An input processing deviceof claim 12 wherein the 2-dimensional coordinates inputted from thepointing device are inputted as a series of coordinates inputted as thepointing device is moved across the display screen.
 14. An inputprocessing device of claim 12 wherein the virtual plane is a curvedplane.
 15. An input processing device of claim 12 wherein thepredetermined input condition is lifting the pointing device from thedisplay screen.
 16. An input processing device of claim 12 wherein theon-virtual plane moving step includes moving the predetermined virtualobject to a position on the virtual plane corresponding to the2-dimensional coordinates detected in the 2-dimensional coordinatedetection step.
 17. An input processing device of claim 12 wherein theon-virtual plane moving step includes displaying the predeterminedvirtual object at positions on the virtual plane corresponding to acontinuous series of the 2-dimensional coordinates detected in the2-dimensional coordinate detection step, and the in-3-dimensional-spacemoving step begins when the 2-dimensional coordinate detection stepdetects the pointing device ceasing the input of the continuous seriesof 2-dimensional coordinates.
 18. An input processing device of claim 12wherein in the display control step includes controlling a display angleof the predetermined virtual object to be projected on the virtual planewhen an input from the pointing device is determined as beingcontinuously inputted.
 19. An input processing device of claim 12 whenan input from the pointing device is determined as being continuouslyconducted, a display angle of the predetermined virtual object isprojected on the virtual plane in the display control step and thedisplay angle is controlled based on the 2-dimensional coordinatesdetected in the 2-dimensional coordinate detection step, and in thein-3-dimensional-space moving step, an initial normal vector of thepredetermined virtual object is set according to the display angle, anda motion trajectory in the virtual 3-dimensional space is calculatedbased on the initial normal vector, wherein the movement of thepredetermined virtual object in the virtual 3-dimensional space is basedon the motion trajectory.
 20. An interactive processing devicecomprising: a display screen adapted to display a projection from avirtual three dimensional space; a pointing device adapted to input tothe device 2-dimensional coordinates corresponding to positions on thedisplay screen; a processor in communication with the display screen andreceiving the input from the pointing device, the processor accessing amemory unit storing on non-transitory tangible media a program executedby the processor to generate a virtual object shown on the displayscreen, the program causing the processor to: set a virtual plane in thevirtual 3-dimensional space; detect the 2-dimensional coordinatesinputted by the pointing device; moving the virtual object on thevirtual plane according to the detected 2-dimensional coordinates;moving the virtual object out of the virtual plane upon detection of apredetermined input from the pointing device, and displaying on thedisplay screen the movements of the virtual object in the virtual planeand out of the virtual plane.